System and methods for multi-dimensional rendering and display of full volumetric data sets

ABSTRACT

A stand-alone platform and a method for the multi-dimensional rendering, display, manipulation, and analysis of full high resolution volumetric data sets. The systems and methods provide the ability to volumetrically render images with extremely high resolution in applications such as medical imaging procedures, digital microscopy such as in use of a confocal microscope, and other areas where extremely large data sets are produced from the imaging process. Certain embodiments of the system and methods produce left and right eye images of the rendered data, for viewing in parallax via a synchronized headset, and the ability to manipulate the data and display of image data easily and in real time.

CROSS-REFERENCE TO RELATED APPLICATIONS/INCORPORATION BY REFERENCE

This U.S. patent application claims priority to and the benefit ofProvisional U.S. Patent Application Ser. No. 60/824,179 filed on Aug.31, 2006, which is incorporated by reference herein in its entirety.

GOVERNMENT SUPPORT

The U.S. Government may have a paid-up license in this invention and theright in limited circumstances to require the patent owner to licenseothers on reasonable terms as provided for by the terms of grant nos.441147 and 441129 from the Fund for the Improvement of Post-SecondaryEducation (FIPSE). The state of Ohio may have a paid up license in thisinvention and the right in limited circumstances to require the patentowner to license others on reasonable terms as provided for by the termsof Technology Action Fund (TAF) grant no. 444505 from the Ohio Board ofRegents of the State of Ohio.

TECHNICAL FIELD

Certain embodiments of the present invention relate to systems andmethods for the real time display and manipulation of extremely largedata sets using real time volume rendering and display of volumetricimages (e.g., stereoscopic volumetric images) using all the data fromextremely large data sets. The systems and methods provide the abilityto volumetrically render images with extremely high resolution inapplications such as medical imaging procedures, digital microscopy suchas in use of a confocal microscope, and other areas where extremelylarge data sets are produced from the imaging process. Certainembodiments of the system and methods produce left and right eye imagesof the rendered data, for viewing in parallax via a synchronizedheadset, and the ability to manipulate the data and display of imagedata easily and in real time.

BACKGROUND

There have been attempts to provide three-dimensional (3D) imaging ofdata to facilitate analysis of the data for various applications,including for use in medical imaging for example. Medical personnel havean important need to see and visualize image data of the structure andcondition of a patient's internal anatomical structures. Such image datamay be generated by non-invasive techniques, such as by imagingmodalities which produce three dimensional (3D) image information. Thesetechniques include, for example, computed tomography (CT), magneticresonance imaging (MRI), positron emission tomography (PET), tomographicgamma scintillation imaging, ultrasound imaging, nuclear medicalimaging/spectroscopy and other techniques.

Using such medical imaging techniques, extremely large data sets areproduced in many cases. The mere volume of data can make it prohibitiveto attempt to use the data in generating display of the image data. Itwould be desirable to use the entire image data set to generate a 3Ddisplay of image data in real time, but prior attempts at volumetricallydisplaying the image data have not produced a system or methods whichallow for such rendering while making it possible to manipulate the datafor evaluation and analysis. The data sets in many cases are simply toolarge to handle in known display systems. Thus, such attempts have beendirected at reducing the size of the data set and interpolating the datato produce a volumetric display without using all the available data.Such attempts produce results of limited value as important structures,details or information may not be seen in the displayed data. Further,it would be desirable to provide a system and methods which allow largedata sets from any type of imaging device, regardless of manufacturerand/or imaging techniques, where volumetric data is produced. Thisincludes, but is not limited to MRI data, ultrasound data, PET scandata, CT scan data, Echo data or any other imaging devices ortechnologies.

It would therefore be desirable to provide a medical image data displaysystem and methods which produces a 3D volumetric representation ofinternal anatomical structures produced from a medical imagingtechnique, using all available data and providing the physician or otherobserver with the ability to manipulate the displayed image datainteractively in real time such that the object may be viewed fromvarious directions and in various modes in real time. It would also bedesirable to generate a real time display of volumetric image data forviewing in a 3D stereoscopic format.

Other environments and applications also generate extremely largevolumetric data sets, such as in the acquisition of image data usingdigital microscopy such as from a confocal microscope, for example, orin the acquisition of seismic data representative of a volume of earthor other medium, weather system data or in other areas. Prior systemsand methods may not be suitable for real-time volume rendering tovisualize a large-scale volume data set, in terms of handling the dataand being cost-effective. Although very expensive and sophisticateddedicated systems may provide certain features, the cost and end useravailability of such systems is prohibitive for general use. It would bedesirable to provide a system which both gives high resolution of thedata with stereoscopic viewing and real time manipulation of the imagedata for effectively visualizing information contained therein in a costeffective solution.

Prior systems are further generally unable to handle time varying datasets, which generally cannot be rendered in a form where the data can beviewed based upon its time relationship with other related data. Forexample, in medical imaging it would be desirable to provide the abilityto image time varying data to perceive differences between the data overtime. Generally, time varying data is not able to be rendered forviewing in a coherent manner, and cannot be rendered in real time toallow a user to interact with the data in the desired way.

Further limitations and disadvantages of conventional, traditional, andproposed approaches will become apparent to one of skill in the art,through comparison of such systems and methods with the presentinvention as set forth in the remainder of the present application withreference to the drawings.

BRIEF SUMMARY

Certain embodiments of the present invention are directed to providingsystems and methods which overcome the limitations of the prior attemptsof displaying three dimensional images, and allows for the real-timevolume and surface rendering of large-scale volume data sets tovisualize three dimensional structures within the data. The system andmethods provide rendering of volumetric display information via astand-alone platform (e.g., a server platform) which may be connected ornetworked to at least one local or remote viewing system, withprocessing being performed at the server side prior to being sent on tothe viewing system(s). The viewing system may be a two dimensional videodisplay device, wherein volumetric data is displayed in, for example,two angularly displaced images which are consecutively viewed via thedisplay device for a true stereoscopic display of the three dimensionalimage data. For example, the 3D image data is viewed by two videodisplay devices associated with a stereoscopic viewing system, such asin the form of stereoscopic glasses to provide projection of thestereoscopic image components sequentially to the right and left eyesrespectively of the viewing in motion parallax. Alternatively, the 3Dvolumetric data may be displayed using passive stereoscopic imagetechnology or monoscopic image technology. A viewer may interact withthe system using a suitable user interface, such as via voice commands,a computer mouse, a keyboard, a touch screen, a tracked virtual realitydevice such as a pinch glove or v-wand for example or any other humaninterface device (HID).

The system, according to an embodiment of the present invention,comprises a multi-dimensional display system for volumetric imaging ofdata sets having a size of one Gigabyte (GB) or more of volumetric data,for example. The system is generally portable, being configured with astand-alone platform (e.g., a stand-alone image rendering server) with apredetermined amount of internal memory. A video driver allows access tofunctions on the video hardware for handling volumetric data is providedto render server side generated monoscopic and passive or activestereoscopic images in conjunction with a rendering software/hardwarebackend associated with the server side. If the images are to be viewedremotely, the video driver may have hooks which poll a local videobuffer to update changes in the represented image data. Otherwise, thelocal video buffer is used to display the images locally. For viewingremotely, the generated images are sent to a remote viewer, which may bea stereoscopic thin client. In an embodiment, the server side is coupledto one or more remote viewers via a network, including a globalinformation network such as the Internet, or other network. From theserver side, a remote framebuffer (RFB) protocol passes video bufferinformation and changes to the remote viewer, where it is buffered viathe viewer client system. From the buffered RFB passive stereoscopicimage data, the remote viewer may provide a passive stereo display foruse on suitable passive stereo display technology via a 3D viewer.Alternatively, the stereoscopic thin client may provide an active stereodisplay, where the passive stereo image is split into left and rightsources via the processing system of the viewer thin client. The leftand right sources are mapped into the left and right stereo buffers, andsupply display information to an active stereoscopic viewing system. Theviewer thin client also may include a HID to allow a user to interactwith the stereo display to manipulate the display information andparameters. Any desired modifications in the displayed image data orparameters are sent from the viewer thin client to the server side, withevents received and sent to the rendering backend for manipulation ofthe display parameters, and modification of the display information sentto the viewer thin client from the server side in real time.

In accordance with an embodiment, the system server computer of thestereoscopic image display system includes the hardware and software torender multi-dimensional image displays using all information inextremely large data sets, such as in the medical imaging, digitalmicroscopy or other environments where such data is generated. Therendered volumetric image data is supplied to a viewer thin client thattherefore does not require processing capabilities to provide suchrendering. The viewer thin client provides the capabilities ofstereoscopic viewing and manipulation of the display parameters. Thesystem may be based on a PC platform, such that it is extremely costefficient while providing the desired real time image displaycharacteristics for data sets having 1 GB of data or more, for example.The system uses the actual volumetric data without interpolation togreatly increase resolution in the displayed image data. The system alsoallows volumetric 4D image display with volumetric image data displayedover a predetermined time period and/or at predetermined time intervals.For example, the volume may be updated so as to provide the ability topage through each time point in the 4D data set, allowing interactionand manipulation with the stereoscopic 4D data (i.e., a beating heart,multi-timepoint MRI, etc.).

It is contemplated, according to certain embodiments, that videoprocessing devices may be developed in the future which would eliminatethe need for various supporting systems to handle the size of the datasets for which the present system is useful. Embodiments of the presentinvention contemplate such improvements and suitable modifications wouldoccur to those skilled in the art based on such improvements to simplifyor reduce the costs associated with the system.

An embodiment of the present invention comprises a stand-alone platformproviding real time high resolution image processing. The stand-aloneplatform includes a first processing unit adapted to automatically reada plurality of volumetric data sets corresponding to a plurality ofmulti-modal or multi-channel volumetric data formats derived from atleast one volumetric data source and extract fully acquired volumetricdata from the data sets. The stand-alone platform further includes asystem bus operationally connected to the at least one processing unit,and at least one graphics processing unit and memory operationallyconnected to the system bus and adapted to receive the fully acquiredvolumetric data from the first processing unit via the system bus. Thestand-alone platform also includes at least one graphics processing unitoperationally connected to the graphics processing unit memory andadapted to render multi-dimensional image data from the fully acquiredvolumetric data in real time. The stand-alone platform further includesa frame compositing and buffering device operationally connected to theat least one graphics processing unit and adapted to buffer frames ofthe rendered multi-dimensional image data and output the buffered framesfor display in real time.

Another embodiment of the present invention comprises a system providingmulti-dimensional rendering, display, and manipulation of full highresolution volumetric data sets. The system includes a stand-aloneplatform adapted to render multi-dimensional image data in real timefrom a plurality of fully acquired volumetric data sets corresponding toa plurality of multi-modal or multi-channel volumetric data formatsderived from at least one volumetric data source. The system furtherincludes at least one display subsystem operationally connected to thestand-alone platform and adapted to display the renderedmulti-dimensional image data in real time. The system also includes atleast one human interface device (HID) operationally connected to thestand-alone platform and adapted to provide interactive real timemanipulation and modification of the rendered multi-dimensional imagedata.

A further embodiment of the present invention comprises a systemproviding multi-dimensional rendering, display, and manipulation of fullhigh resolution volumetric data sets. The system includes a stand-aloneplatform adapted to render multi-dimensional image data in real timefrom a plurality of fully acquired volumetric data sets corresponding toa plurality of multi-modal or multi-channel volumetric data formatsderived from at least one volumetric data source. The system furtherincludes at least one remote thin client viewer operationally connectedto the stand-alone platform and adapted to receive the renderedmulti-dimensional image data from the stand-alone platform in real timeusing a remote frame buffer (RFB) protocol, and adapted to display therendered multi-dimensional image data stereoscopically in real time. Thesystem also includes at least one human interface device (HID)operationally connected to the remote thin client viewer and adapted toinitiate interactive real time manipulation and modification of therendered multi-dimensional image data upon activation of the HID by auser. The manipulation and modification is accomplished when the remotethin client viewer sends an event command to the stand-alone platform inreal time in response to the HID activation. The renderedmulti-dimensional image data is updated by the stand-alone platform inreal time in response to the event command and the stand-alone platformsends the updated rendered multi-dimensional image data to the remotethin client viewer in real time using the RFB protocol. The updatedrendered multi-dimensional image data is displayed by the remote thinclient viewer in real time.

Another embodiment of the present invention comprises a method for themulti-dimensional rendering, display, manipulation, and analysis of fullhigh resolution volumetric data sets. The method includes automaticallyloading a plurality of fully acquired volumetric data sets correspondingto a plurality of multi-modal or multi-channel volumetric data formatsderived from at least one volumetric data source into a stand-aloneplatform. The method further includes processing the plurality of fullyacquired volumetric data sets within the stand-alone platform to extractfully acquired volumetric data from the data sets and to generate atleast one rendered volume image in real time from the extracted data.The method also includes displaying at least one rendered volume imageon a display subsystem in real time and modifying at least one displayparameter of the displayed rendered volume image using at least onehuman interface device (HID) operationally connected to the stand-aloneplatform. The method further includes updating the at least one renderedvolume image within the stand-alone platform in real time in response tothe at least one modified display parameter and displaying the updatedat least one rendered volume image on the display subsystem in realtime.

A further embodiment of the present invention comprises a stand-aloneplatform providing real time high resolution image processing. Thestand-alone platform includes means for automatically reading aplurality of volumetric data sets corresponding to a plurality ofmulti-modal or multi-channel volumetric data formats derived from atleast one volumetric data source and extracting fully acquiredvolumetric data from the data sets. The stand-alone platform furtherincludes means for rendering multi-dimensional image data from the fullyacquired volumetric data in real time, and means for buffering frames ofthe rendered multi-dimensional image data and outputting the bufferedframes for display in real time.

Another embodiment of the present invention comprises a system providingmulti-dimensional rendering, display, and manipulation of full highresolution volumetric data sets. The system includes means for renderingmulti-dimensional image data in real time from a plurality of fullyacquired volumetric data sets corresponding to a plurality ofmulti-modal or multi-channel volumetric data formats derived from atleast one volumetric data source. The system further includes means fordisplaying the rendered multi-dimensional image data in real time andmeans for providing interactive real time manipulation and modificationof the rendered multi-dimensional image data.

A further embodiment of the present invention comprises a systemproviding multi-dimensional rendering, display, and manipulation of fullhigh resolution volumetric data sets. The system includes means forrendering multi-dimensional image data in real time from a plurality offully acquired volumetric data sets corresponding to a plurality ofmulti-modal or multi-channel volumetric data formats derived from atleast one volumetric data source. The system further includes means forremotely receiving and displaying the rendered multi-dimensional imagedata in real time. The system also includes means for remotelyinitiating interactive real time manipulation and modification of therendered multi-dimensional image data. The manipulation and modificationis accomplished when an event command is received in real time at themeans for rendering in response to the initiating. The renderedmulti-dimensional image data is updated by the means for rendering inreal time in response to the event command. The means for renderingsends the updated rendered multi-dimensional image data to the means forremotely receiving and displaying in real time, and the updated renderedmulti-dimensional image data is displayed in real time.

Another embodiment of the present invention comprises a stand-aloneplatform providing real time high resolution image processing. Thestand-alone platform includes means for automatically reading aplurality of volumetric data sets corresponding to a plurality ofmulti-modal or multi-channel volumetric data formats derived from atleast one volumetric data source and extracting fully acquiredvolumetric data from the data sets. The stand-alone platform alsoincludes means for rendering multi-dimensional image data from the fullyacquired volumetric data in real time. The stand-alone platform furtherincludes means for automatically re-configuring the means for renderingbased on at least one characteristic of the volumetric data sets. Thestand-alone platform further includes means for buffering frames of therendered multi-dimensional image data and outputting the buffered framesfor display in real time.

These and other advantages and novel features of the present invention,as well as details of illustrated embodiments thereof, will be morefully understood from the following description and drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates an exemplary functional block diagram of a firstembodiment of a real time volume rendering system;

FIG. 2 illustrates an exemplary functional block diagram of a secondembodiment of a real time volume rendering system;

FIG. 3 illustrates an exemplary functional block diagram of anembodiment of the software of the stand-alone platform of the system ofFIG. 1 or FIG. 2;

FIG. 4 illustrates an exemplary schematic block diagram of an embodimentof the hardware of a stand-alone platform within a system;

FIG. 5 illustrates a flowchart of an exemplary embodiment of a method500 for the multi-dimensional rendering, display, manipulation, andanalysis of full high resolution volumetric data sets;

FIG. 6 illustrates an exemplary functional data flow diagram of thevolume rendering operation of the volume rendering system of FIG. 1;

FIG. 7 illustrates an exemplary embodiment of a displayed renderedvolume image generated by the volume rendering system of FIG. 1;

FIG. 8 illustrates an exemplary first embodiment of a functional dataflow diagram showing the various plurality of data types and modalitiesthat may be handled by the stand-alone platform of FIG. 3 and FIG. 4 forvolumetric image rendering;

FIG. 9 illustrates an exemplary functional data flow diagram showingvarious types of automated segmentation that may be provided by thevolume rendering system of FIG. 1 or FIG. 2; and

FIG. 10 illustrates an exemplary second embodiment of a functional dataflow diagram showing the various plurality of data types and modalitiesthat may be handled by the stand-alone platform of FIG. 3 and FIG. 4 forvolumetric image rendering.

DETAILED DESCRIPTION

FIG. 1 illustrates an exemplary functional block diagram of a firstembodiment of a real time volume rendering system 10 according to anexample of the invention, for use in generating and displayingvolumetric image data. A dataset 12 comprises a multimodal 3D/4Dvolumetric dataset, such as may be produced in medical imaging systems,digital microscopy imaging, seismic exploration or other systems wherean extremely large amount of volumetric data is generated. Although thesystem 10 is capable of volumetric image display using smaller datasets,it is particularly useful for very large datasets of one GB or more.More particularly, the system 10 is usable to image in real time,datasets of three GB or more. In accordance with an embodiment, theimaging hardware and software will automatically adapt to handle datasets of significantly different byte sizes. With datasets of large size,many prior systems have not been able to effectively process and renderimage data from the entire dataset, instead resorting to interpolationtechniques or surface rendering techniques to form a pseudo-volumetricimage display, and not using all of the data acquired and available forimaging.

Although many such datasets may require no preprocessing for effectivelyrendering a volumetric image display from the entire dataset, the datais optionally preprocessed in the image generation system 14. The imagegeneration system 14 may therefore include data preprocessing modulesfor any desired preprocessing of the data, such as automaticsegmentation, clipping, compression, conversion of format,deconvolution, resampling, normalizing, or the like at 16. It should berecognized that as it is desired to utilize all available data, datacompression or other techniques that result in a loss of data are notnecessary. The raw or preprocessed data is input to imaging software at18, and a software and/or hardware based volume rendering system 20 isused to generate volumetric multi-dimensional image data. The volumerendering system 20 may be adaptive such that, for large data sets,image processing hardware is heavily relied upon and, for smaller datasets, image processing software is relied upon more than imageprocessing hardware. Based on the characteristics of the data, the useof hardware and software is optimized for processing the data andrendering volumetric images in real time. A hardware based stereoscopicimage is generated at 22 from the volume rendering system 20.

Once the volumetric image data is used to generate stereoscopic imagedata, this information is selectively coupled to one or more localstereoscopic display systems at 24 and/or to one or more remotestereoscopic display systems at 26. The at least one remote system 26may be coupled via any suitable network 28, such as the Internet forremote stereoscopic user interaction, analysis and data and displaymanipulation. Further, it should be understood that one or more remotesystems 26 may be at the same or different locations. In this way, thesystem and methods of the invention may allow multiple users to view theimage data simultaneously, either at the same or different locations,allowing the users to collaborate and interact with the image data in avery efficient and meaningful manner. One or more local systems 24 mayalso be used for stereoscopic user interaction, analysis and data anddisplay manipulation.

As will be described further with respect to FIG. 2, the system 10 of anembodiment of the present invention, may divide processing and handlingof the very large datasets apart from the stereoscopic display andmanipulation functions with which the user interacts. FIG. 2 illustratesan exemplary functional block diagram of a second embodiment of a realtime volume rendering system. The image generation system 14 shown inFIG. 1 may comprise a shared display server 30 having various hardwareand software functions for generating stereoscopic image displayinformation for use by the local and/or remote display system 24 or 26as shown in FIG. 1. As seen in FIG. 2, the server 30 has a renderingsoftware/hardware backend 32 which is adapted to render volumetric imagedata from volumetric datasets, whether such datasets are representativeof raw volumetric data or preprocessed volumetric data.

As an example, the rendering software/hardware backend 32 is a videoprocessing unit having predetermined processing capabilities forrendering video image data from the large datasets contemplated. Asuitable processing unit for use in the invention is a graphicsprocessing unit produced by Nividia or the VolumePro 1000D hardwarerendering PCIX graphics card equipped with 4 GB internal memory producedby TeraRecon, Inc. of San Mateo, Calif. The volume rendering video cardmay be used in conjunction with volume rendering software/hardware ifdesired, and customized software may be provided in an embodiment of theinvention. Alternatively, the Amira visualization software packageproduced by TGS, Inc. of Richmond, Tex. may be usable.

The rendering software/hardware backend 32 provides for generation ofserver side passive stereoscopic images at 34 which are stored in alocal video buffer and selectively coupled to a stereoscopic viewer thinclient 50. For real time viewing of the multi-dimensional passivespectroscopic images, the video driver associated with the renderingsoftware/hardware backend 32 may be configured with hooks to poll thelocal video buffer at 36 for any requested changes in the displayparameters, which are immediately implemented by the backend 32 to bestored in the local video buffer for selectively passing to the viewerclient. In an embodiment, the image display generated by the server side30 is coupled to the viewer client 50 by means of a remote framebufferprotocol (RFB) at 38 to pass the image data to the thin client 50 forviewing by a user. The system provides the ability to render volumetricimage data and manipulate such image data in real time.

If desired or necessary based on the format or other characteristics ofthe volumetric image data, the data may be preprocessed in the serverside system 30 before or after rendering of the stereoscopic volumetricimages. Preprocessing may use various schemes for converting the datainto a different format, modifying the data into a different form, forsegmentation or partitioning of the data, or other suitable processingsteps to prepare the data for rendering, communication, or otherpurposes. Such preprocessing steps will depend on the nature of thevolumetric data, and particular hardware and/or software components ofthe system. For example, compression/decompression encoding schemes maybe applied to the data if desired, and filtering or automaticsegmentation of the data may be performed prior to passing the data tothe viewer client. For example, the software may provide for imagesegmentation via an editor for real-time 3D control of the segmentationprocess.

The backend 32 may also support time-dependant data for truefour-dimensional (4D) visualization of the volumetric data. The abilityto provide 4D image display may be coupled with pre-processingtechniques to automatically present the volumetric images to the viewingclient 50 in a predetermined form with time-dependant images presentedfor display with the time-dependency selected by a user via a suitableHID.

The viewer client 50 is a thin client that does not need substantialprocessing power for rendering the volumetric spectroscopic images, butinstead receives the image data after such processing has been performedon the server side 30. As seen in FIG. 2, the viewer client 50 isprovided with the RFB passive spectroscopic image data and stores suchimage data in a suitable local buffer at 52. From the buffered images at52, the viewing client 50 may be configured to display the images withpassive stereo technology at 54.

Alternatively, or in addition, the passive stereo image data is splitinto left and right eye source images at 56 for an active stereodisplay. In this event, the left/right sources are then mapped into aleft source stereo buffer at 58 and a right source stereo buffer at 60.With either the passive stereo display or an active stereo display, theimages are then sent to a passive or active stereo viewer 62.

In an embodiment, a passive viewer may be a high resolution CRT, LCDdisplay or other suitable display device viewed with a suitable 3Dviewing system such as stereo-viewing glasses worn by a user. For anactive stereo display, the left and right stereo images are synchronizedand sent to an active stereo display worn by the user, such as stereodisplay glasses on which the left and right images are sequentiallydisplayed. For example, a suitable active display system is theCrystalEyes active stereo display system and glasses, such as sold byVrlogic GmbH for example. Such systems may use a quad-bufferedframe-sequential stereo display mode used to drive LCD shutter glasseswith a CRT display, LCD, projection or other suitable system.Quad-buffered frame-sequential stereo may be provided with the viewingclient 50 for use with an active display, to provide separate left andright eye frame buffers for such an active display, and providing truestereoscopic viewing of the image data.

The user, via a suitable HID associated with client 50, also has thecapability to manipulate the images being coupled to the viewer client50 in real-time. As used herein, the term “real time” refers to analmost immediate response, such as a user perceiving an almost immediateresponse when the user manipulates the image or performs image analysisvia the HID. The user interacts with the stereo display using aninterface such as a keyboard, mouse, a tracked virtual reality devicesuch as a pinch glove or v-wand for example or any other suitable HID at64. As an example to facilitate simplified use of the system, atouchscreen interface or the like may be provided in association withthe viewer client 50, to select areas of the displayed images to bemanipulated or to select items from displayed menus for alternativecommand input and display manipulation. The HID events are sent to therendering server 30 and received at 40. Such events are sent to thebackend 32 to manipulate the display parameters and forward the videobuffer changes to the viewer client 50 in real time.

As mentioned previously, the video driver associated with the server 30polls the local video buffer for changes at 36 before sending on thegenerated passive stereoscopic images to the viewer client 50. Thedisplay parameters may be modified in any variety of ways by the user toachieve the desired display of volumetric spectroscopic images toevaluate, analyze, measure or otherwise use the images for gainingdesired information form the volumetric data. In this way, the user maymanipulate the stereo images provided to the viewer client to providesimple and desired visual navigation through the images.

Such manipulation of display parameters allow the user to rotate theimage, zoom in or out, perform dimensional measurements betweenstructures or along surfaces, segment image information, change color oropacity transfer functions, or a wide variety of other variation in thedisplayed images. This provides the user with a real-time visualnavigation interface for use and access to the resolution capabilitiesof the server generated passive stereoscopic images using the extent ofthe large volume of volumetric data supplied to the system. Theinterface thus allows for volume navigation of the data throughreal-time interaction with the user(s).

For use in viewing medical imaging data for example, the system of anembodiment of the present invention provides for real-time volumerendering and manipulation for viewing on the viewer client for advanceddiagnosis, analysis, or operational tool in the detection or treatmentof disease. Similarly, in other volumetric systems, such as digitalmicroscopy, weather system imaging, seismic or other volumetricgeological data, video entertainment or virtual reality systems, theability to utilize all available data to achieve very high resolutionwhile providing for real time manipulation by a user in real timegreatly enhances the ability to effectively use and/or evaluate suchdata.

Turning to FIG. 3, operation of an embodiment of the system will bedescribed. FIG. 3 illustrates an exemplary functional block diagram ofan embodiment of the software of the stand-alone platform, such as acomputer-based platform, of the system of FIG. 1 or FIG. 2. The imagingserver as previously described may have auxiliary processing software at70, to which the dataset, being any 2D, 3D or 4D dataset, is supplied tothe system at 72. Pre-processing may be performed if desired or needed,by semi-automated data preprocessing software at 74. Either the raw dataor preprocessed data is then forwarded to the imaging software system 80and a data input system at 82. The system 82 supports any desired dataformat or structures for handling in the system, such as 3D or 4D vendorspecific data forms including image stacks, volumes, surfaces VRML,DICOM, raw, image stacks, VTK or any other data. That is, the system canautomatically read or load any volumetric data to be imaged. As used inthis context, the term “automatically” means without significant userintervention and without having to first convert the data to some commonformat. The data is then processed or extracted if needed via a dataprocessing/extraction sub-system 84, where any desired processing of thedata may be performed prior to image generation.

In the embodiment shown in FIG. 3, such processing may be of differentcharacteristics. A clipping sub-system 86 may provide desired clippingfeatures and manipulation of the data. For example, real-timeinteractive data clipping procedures can be selected by the user, suchas for data clipping planes, isolation of data chunks or groups, volumeediting, changing of the physical and virtual dimensions in thedisplayed images, skewing of the data, or data resampling, as examples.A segmentation sub-system 88 may be provided for performing processessuch as automatic surface generation, threshold and interactivesegmentation, volume or distance measurement and/or 3D extractiontechniques as examples.

Further, although embodiments of the present invention may alleviate theneed to deconvolve data for handling thereof due to the large volume ofdata, it may still be desirable to perform deconvolution processing forvarious applications at 90. Such processing may include correction ofz-drop or point spread function based deconvolution, for example. Ifdata processing is performed at 84, the processed data is then suppliedback to the data input module at 82 for subsequent image generation viaan image generation module 92.

The image generation module 92 may perform various functions as desiredfor a particular application, including for example, volume rendering at94. This can include 2D and 3D texture mapped volume renderingtechniques, mean intensity projection, control over data transparencyand color, the use of customizable look up tables, real-time imagemodification and manipulation, software access to hardware volumerendering functions for massive datasets, or any other desired imagegeneration characteristics.

The image generation module 92 may also provide for surfacerepresentation generation at 96. For example, procedures such asautomatic iso-surface extraction, the use of semi-transparent surfaces,back/front surface control, surface color/texture selection, surfacereflection/lighting segmented object viewing or the creation of texturemapped surfaces may be provided.

The image generation module 92 may also perform other display generationprocessing such as for the display of many objects concurrently, for theoverlay of data in images and/or for manipulating display parameters inreal-time. Additional features, such as the provision and use of scenebased modifiers may also be provided. These may include for example,global lighting features, background colors, camera position, stereo 3Dreal-time output type or buffer options.

Also, the system 92 may provide for features for animation of the data,such as animating the camera, object or any other display parameters,manipulation of the 3D image and/or the interactive animation of 4Ddata, the output of stereo- or mono-scopic movies and snapshots of theimage data for example. Once generated the rendered image data is outputat 100 in a desired form. These forms may include for example,monoscopic, passive stereo, interlaced stereo, active stereo, tileddisplays and virtual reality displays. As previously mentioned, theimaging software system 80 interacts with the hardware systems on theserver side via a quad buffered OpenGL driven backend for active stereoviewing as an example, that has access to the graphics processing unitat 100. For passive stereo viewing, such hardware is not necessary.

FIG. 4 illustrates an exemplary schematic block diagram of an embodimentof the hardware of a stand-alone platform 401 within a system 400. Thestand-alone platform 401 includes a processing unit 410 adapted to readone or more volumetric data sets corresponding to one or more modes orchannels of volumetric data formats, and to extract volumetric data fromthe data sets. The volumetric data sets are derived from at least onevolumetric data source 495 such as an optical disk or compact disk, forexample. A mode, as used herein, refers to an imaging mode such as, forexample, X-ray or MRI. The term “multi-modal” as used herein refers toone or more modes, or the capability to process data from one or moremodes. A channel, as used herein, refers to a channel of data such as,for example, an immunohistochemically stained color channel of dataobtained from a confocal microscope. The term “multi-channel” as usedherein refers to one or more channels, or the capability to process datafrom one or more channels.

The processing unit 410 is adapted to extract fully acquired volumetricdata from the data sets. As used herein, the term “fully acquired”refers to all of the volumetric data that was acquired upon acquisitionsuch as, for example, via a medical imaging system, as opposed tointerpolated data or sampled data forming a reduced data set. As aresult, the fully acquired volumetric data includes all of theinformation in, and the highest resolution of, the data set at the timeof acquisition.

The stand-alone platform 401 also includes a system bus 420 providingcommunication and data transfer between certain various subsystems andcomponents of the stand-alone platform 401. For example, the stand-aloneplatform 401 further includes a memory 415 operationally connected tothe processing unit 410 via the system bus 420. The processing unit 410and the memory 415 are used, for example, to extract and spatiallyregister multiple modes of the volumetric data.

The stand-alone platform 401 includes a graphics processing unit memory430 operationally connected to the system bus 420 and adapted to receivethe fully acquired volumetric data from the processing unit 410 via thesystem bus 420. The stand-alone platform 401 further includes at leastone graphics processing unit 440 and/or 441 operationally connected tothe graphics processing unit memory 430 and adapted to rendermulti-dimensional (e.g., 2D, 3D, 4D) image data from the fully acquiredvolumetric data in real time. The graphics processing unit 440 may bededicated to volume rendering, for example. The graphics processing unit441 may be dedicated to surface rendering, for example. Alternatively,both of such functions may be performed by a single graphics processingunit.

A frame compositing and buffering device (hardware or software) 450 isoperationally connected to the graphics processing unit 440 and/or 441and is adapted to buffer frames of the rendered multi-dimensional imagedata and output the buffered frames for display in real time. Thedisplay device 496, for example, may be operationally connected to theframe compositing and buffering device 450 and is adapted to display therendered multi-dimensional image data in real time. The display device496 may or may not be considered part of the stand-alone platform 401.The display device 496 and/or 497 may be used with passive or activestereoscopic eyewear 498 or 499, for example, for viewing renderedstereoscopic images as described previously herein.

The processing unit 410 may be further adapted to co-register the fullyacquired volumetric data to a common spatial coordinate system beforepassing the fully acquired volumetric data to the graphics processingunit memory. For example, if the volumetric data sets include magneticresonance imaging (MRI) data from a first medical imaging mode (i.e.,MRI) in one data set and positron emission tomography (PET) data from asecond medical imaging mode (i.e, PET) in another data set, theprocessing unit 410 is capable of reading both data sets, extracting thefully acquired volumetric data from both data sets, and registering boththe MRI and PET volumetric data to, for example, a common spatial and/ortemporal coordinate system (e.g., 3D or 4D). As a result of suchregistration, voxels of both the MRI data and the PET data will beproperly spatially and/or temporally represented with respect to eachother.

The stand-alone platform 401 also includes at least one human interfacedevice (HID) 460 operationally connected to the system bus 420 andadapted to interactively initiate real time manipulation, modification,and analysis of the rendered multi-dimensional image data. The HID 460may include, for example, any of a computer mouse, a joy stick, atouch-screen panel, a keyboard, a voice-activated command device, atracked virtual reality device such as a pinch glove or v-wand forexample or any other type of HID useful for providing real timeinteraction with the displayed image data. As previously stated herein,real time refers to providing an almost immediate response. Inaccordance with another embodiment of the present invention, the HID 460is not considered part of the stand-alone platform 401 but is insteadconsidered a system component which interfaces to the stand-aloneplatform 401 via any one of a plurality of possible interfaces providedby the stand-alone platform 401.

In accordance with an embodiment, the stand-alone platform 401 includesa network interface 470 operationally connected to the system bus 420and adapted to communicatively interface with a remote storage device475 and a remote display subsystem 497. For example, the networkinterface 470 may provide connection to the Internet allowing renderedmulti-dimensional image data to be displayed at the remote displaydevice 497 and allowing volumetric data sets to be read from the remotestorage device 475.

In accordance with an embodiment of the present invention, thestand-alone platform 401 includes a local storage device 480 capable ofstoring the volumetric data sets read from the volumetric data source495. The volumetric data sets may be loaded into the local storagedevice 480 or the remote storage device 475 from the volumetric datasource 495. The stand-alone platform 401 may further include a powersupply 490, universal serial bus (USB) inputs 491, and an optical drive492, for example. The stand-alone platform 401 may include otherPC-related hardware as well, in accordance with various embodiments.

A system configuration 400, in accordance with an embodiment of thepresent invention, includes the stand-alone platform 401 adapted torender multi-dimensional image data in real time from a plurality offully acquired volumetric data sets corresponding to a plurality ofmulti-modal or multi-channel volumetric data formats derived from atleast one volumetric data source 495, at least one display subsystem 496operationally connected to the stand-alone platform 401 and adapted todisplay the rendered multi-dimensional image data in real time, and atleast one HID 460 operationally connected to the stand-alone platform401 and adapted to provide interactive real time manipulation andmodification of the rendered multi-dimensional image data. In accordancewith an embodiment, the HID 460 is further adapted to provideinteractive real-time quantitative analysis of the renderedmulti-dimensional image data. The display subsystem 496 may include ahigh resolution display (e.g., a CRT) operationally connected to thestand-alone platform 401 at the frame compositing and buffering device450 for displaying the rendered multi-dimensional image data.

The system configuration 400 may further include stereoscopic eyewear498 which may be active or passive. If active, the stereoscopic eyewear498 is wired or wirelessly connected to and synchronized with thedisplay subsystem 496 and adapted to provide active stereoscopic viewingof the displayed rendered multi-dimensional image data in real time to auser wearing the active stereoscopic eyewear 498. If passive, thestereoscopic eyewear 498 is adapted to provide passive stereoscopicviewing of the displayed rendered multi-dimensional image data in realtime to a user wearing the passive stereoscopic eyewear 498.

Another system configuration, in accordance with another embodiment ofthe present invention, includes the stand-alone platform (e.g., 401 or30) adapted to render multi-dimensional image data in real time from aplurality of fully acquired volumetric data sets corresponding to aplurality of multi-modal or multi-channel volumetric data formatsderived from at least one volumetric data source 495, and at least oneremote thin client viewer 50 operationally connected to the stand-aloneplatform and adapted to receive the rendered multi-dimensional imagedata from the stand-alone platform in real time using a remote framebuffer (RFB) protocol, and adapted to display the renderedmulti-dimensional image data in real time. The system configuration alsoincludes at least one HID at 64 operationally connected to the remotethin client viewer 50 and adapted to initiate interactive real timemanipulation and modification of the rendered multi-dimensional imagedata upon activation of the HID by a user. The manipulation andmodification is accomplished when the remote thin client viewer sends anevent command to the stand-alone server platform 401 in real time inresponse to the HID activation. The rendered multi-dimensional imagedata is updated by the stand-alone platform in real time in response tothe event command and the stand-alone platform sends the updatedrendered multi-dimensional image data to the remote thin client viewerin real time using the RFB protocol. The updated renderedmulti-dimensional image data is displayed by the remote thin clientviewer in real time.

Again, the system configuration may include active or passivestereoscopic eyewear such that the active stereoscopic eyewear is wireor wirelessly connected to and synchronized with the thin client viewer.In accordance with an embodiment, the thin client viewer 50 includes abuffer memory device at 52 for accepting the rendered multi-dimensionalimage data from the stand-alone platform, and a high resolution displayoperationally connected to the buffer memory device for displaying therendered multi-dimensional image data.

In accordance with an embodiment, the HID at 64 is further adapted toinitiate interactive real time quantitative analysis of the renderedmulti-dimensional image data upon further activation of the HID by auser. The quantitative analysis is accomplished when the remote thinclient viewer 50 sends a further event command to the stand-aloneplatform in real time in response to the further activation of the HID.The quantitative analysis is performed by the stand-alone platform inreal time in response to the further event command to generatequantitative metrics. The stand-alone platform sends the quantitativemetrics to the remote thin client viewer in real time using the RFBprotocol. The quantitative metrics is displayed by the remote thinclient viewer along with the rendered multi-dimensional image data inreal time. In an example, the quantitative analysis may also be used tocompare temporal changes in metrics/parameters of the data in 3D and/or4D data sets. Also, in multi-modal data sets as will be described inmore detail, it is possible to compare co-localization of data (ie., inmicroscopy, the presence of multiple proteins in a cell with eachrepresented by a single channel or changes in medical image data withina specific anatomical regions with the data).

FIG. 5 illustrates a flowchart of an exemplary embodiment of a method500 for the multi-dimensional rendering, display, manipulation, andanalysis of full high resolution volumetric data sets. In step 510,automatically load a plurality of fully acquired volumetric data setscorresponding to a plurality of multi-modal or multi-channel volumetricdata formats derived from at least one volumetric data source into astand-alone platform. In step 520, process the plurality of fullyacquired volumetric data sets within the stand-alone platform to extractfully acquired volumetric data from the data sets and to generate atleast one rendered volume image in real time from the extracted data. Instep 530, display the at least one rendered volume image on a displaysubsystem in real time. In step 540, modify at least one displayparameter of the displayed rendered volume image using at least onehuman interface device (HID) operationally connected to the stand-aloneplatform. In step 550, update the at least one rendered volume imagewithin the PC-based sever platform in real time in response to the atleast one modified display parameter. In step 560, display the updatedat least one rendered volume image on the display subsystem in realtime. The at least one rendered volume image may include a stereoscopicpair of volumetric images. The processing step may include registeringthe extracted fully acquired volumetric data to a common spatial and/ortemporal coordinate system.

In accordance with an embodiment, modifying at least one displayparameter results in at least one of an orientation change of the atleast one rendered volume image, a color transfer function change of theat least one rendered volume image, an opacity transfer function changeof the at least one rendered volume image, and a segmenting of the atleast one rendered volume image.

The method 500 may include other steps as well. For example, the method500 may further include selecting and displaying an iso-surface of therendered volume image in real time. The method 500 may also includeperforming real time navigation through the rendered volume image.

The method 500 may further include performing real time interactivequantitative analysis of the rendered volume image. For example, a usermay designate a first point within the rendered volume image using theHID. The user may then designate a second point in the rendered volumeimage using the HID. A straight line or curved distance may then becomputed between the first point and the second point in real time,giving the user a true measurement of the actual distance between thetwo designated points. Similarly, the method 500 may also includedrawing a probing line or curve through the rendered volume image inreal time such that various measurements may be made along the probingline or curve, such as an iso-surface in the data. As another example, auser may segment out a portion of the rendered volumetric image usingthe HID in real time and command the system to compute a volume orsurface area of the segmented portion in real time.

In accordance with an embodiment, the stand-alone platform 401 iscapable of automatically re-configuring a combination of active imageprocessing and rendering hardware and software components based on atleast one characteristic of the volumetric data sets to be processed.The at least one characteristic may include one or more of a byte sizeof the volumetric data sets, a number of the volumetric data sets, anumber of different modalities included in the volumetric data sets, anda number of dimensions included in the volumetric data sets, or otherdata set characteristics for example.

For example, for very large data sets, the vast majority of the imageprocessing rendering may be performed in hardware (e.g., a graphicsprocessing unit), with software used simply for accessing the hardwarecapabilities in a user friendly and efficient manner. For smaller datasets, the vast majority of the image processing rendering may beperformed in software using the processing unit 410, for example.Furthermore, image process rendering may be allocated between hardwarecomponents. For example, volume rendering may be handled by a firstvideo card 440 and surface rendering may be handled by a second videocard (GPU) 441. As a result, the stand-alone platform 401 is flexible inhow rendering is accomplished in dependence on the nature orcharacteristics of the actual data to be processed. The determination ofthe hardware and software functions relating to a particular data set orsets, may be optimized automatically, or without significant userintervention, based upon processing characteristics of the hardwareand/or functions of the software.

FIG. 6 illustrates an exemplary functional data flow diagram 600 of thevolume rendering operation of the volume rendering system 10 of FIG. 1.A user selects input data (i.e., at least one volumetric data set) at601. At 602, the volumetric data set is automatically identified andread without the user having to tell the system the format or any otherinformation about the volumetric data set. At 603, information isextracted from the read volumetric data sets (e.g., patient/sample nameand information at 604 and data dimensions and voxel size at 605). Adata specific input routine may be performed on volumetric data sets at606 such as, for example, registering multiple modalities (modes) of thevolumetric data sets to a common spatial and/or temporal coordinatesystem.

At 607, a user may interact with the system via a human interface device(HID) in order to perform, for example, data clipping at 608, pixelopacity modification at 609, pixel color modification at 610,iso-surface color modification at 611, iso-surface value modification at612, iso-surface opacity modification at 613, data object positioning at617, and camera zooming at 621. Pixel color and opacity may be initiallydefined by pre-defined transfer functions at 614 in the form of colormaps and opacity maps.

Volume rendering is performed at 615 and surface rendering is performedat 616. Frame compositing and buffering is performed at 618. Displayingof image data is performed at 619. Quantitative metrics may be obtainedfrom the displayed image data at 620.

FIG. 7 illustrates an exemplary embodiment of a displayed renderedvolume image 710 generated by the volume rendering system of FIG. 1. Theimage 710 is a 3D volumetric section of a brain. A color transferfunction display mapping is shown at 720 and an opacity transferfunction display mapping is shown at 730. These mappings may be modifiedby the user via the HID, such as by movement of a slider tool 732 asrepresented for each transfer function 730, which is selectively movablebetween minimum and maximum settings. Other rendering and displayoptions (e.g., stereo options and image modification options) are shownat 740. A user may interact via the HID to rotate the image 710, zoom inor out on the image 710, segment the image 710, modify the color and/oropacity transfer functions applied to the image 710, etc.

FIG. 8 illustrates an exemplary first embodiment of a functional dataflow diagram 800 showing the various plurality of data types andmodalities that may be handled by the stand-alone platform of FIG. 3 andFIG. 4 for volumetric image rendering. Three volumetric data sets of thesame modality are shown at 801, 802, and 803 which are successive datasets acquired in time. The data sets are temporally registered to eachother at 804 and a composite 4D (3 spatial dimensions and a timedimension) volumetric data set is formed at 805. Similarly, a single 3Dvolumetric data set corresponding to a single modality at a single timeis shown at 806.

A volumetric data set of a first modality is shown at 807 and avolumetric data set of a second modality is shown at 808. The data setsare spatially registered to each other at 809 and a composite 3Dmulti-modality volumetric data set is formed at 810. In this example,the fully acquired data sets of each of the modalities is used in thecomposite data set at 810.

Patient information may be extracted from any of the data sets (at 805,806 and 810) at 811, and automated feature extraction and identificationmay be performed at 812. Similarly, automated segmentation of the datasets may be performed at 813 and data clipping may be performed at 814.

Stereoscopic volume rendering may be performed at 815. Surfaceextraction may be performed at 816 and stereoscopic surface rendering ofthe extracted surface may be performed at 817. Stereoscopic framecompositing and buffering is performed at 818. Stereoscopic displayingis performed at 819. Quantitative metrics may be obtained from thedisplayed stereoscopic image data at 820.

FIG. 9 illustrates an exemplary functional data flow diagram 900 showingvarious types of automated segmentation 813 that may be provided by thevolume rendering system of FIG. 1 or FIG. 2. Data normalization isperformed on the volumetric data at 901 (e.g., brightness correction,skew correction, stretch, re-sampling, shear, rotate).

In a first type of automated segmentation, volumetric data is importedinto a common stereotactic space at 902 and is compared and segmentedbased on a pre-defined atlas or shapes at 903. Only those segments ofvolumetric data 910 corresponding to the atlas are extracted at 905. Ina second type of automated segmentation, statistical algorithmicsegmentation is performed on the volumetric data at 904, and multiplerepresentative sub-data sets 920 are extracted at 905. In a third typeof automated segmentation, guided/automated intensity based segmentationis performed on the volumetric data at 906 based on user input selectionof intensity at 909, and multiple representative sub-data sets 930 areextracted at 905.

Volume rendering of the segmented data is performed at 907 and imagedisplaying is performed at 908. A displayed rendered composite image ofthe segmented data 910 is shown at 940, a displayed rendered compositeimage of the segmented data 920 is shown at 950, and a displayedrendered composite image of the segmented data 930 is shown at 960. Foreach of these transfer functions characteristics, an example image andthe results of the application of the transfer function is shown in FIG.9. It should also be recognized that segmented data can be overlayed onthe original data to provide further insights into relationships betweensegmented and nonsegmented data.

FIG. 10 illustrates an exemplary second embodiment of a functional dataflow diagram 1000 showing the various plurality of data types andmodalities that may be handled by the stand-alone platform of FIG. 3 andFIG. 4 for volumetric image rendering. A 4D volumetric data set is shownat 1001, a multi-modal 3D volumetric data set is shown at 1002, amulti-channel 3D volumetric data set is shown at 1003, and a single mode3D volumetric data set is shown at 1004.

The 4D volumetric data set is converted to registered temporal data at1005. A temporal update rate of the registered temporal data is modifiedby a user at 1009 via the HID at 1008. For example, the 4D volumetricdata set may correspond to 4D MRI data. Five temporal images of the MRIdata are shown at 1020. A displayed rendered composite 4D image of theMRI data is shown at 1030.

The multi-modal 3D volumetric data sets are converted into sub-data setsat 1006 corresponding to the different modalities. Opacity or colors ofthe sub-data sets may be modified by a user at 1007 via the HID at 1008via the appropriate transfer functions for example. As another example,the multi-modal 3D volumetric data sets may correspond to a CT data setshown as an image at 1040 and a PET data set shown as an image at 1050.A displayed rendered composite 3D image of the CT and MRI data is shownat 1060.

As a further example, the multi-channel 3D volumetric data sets maycorrespond to a first color of stained tissue imaged by a confocalmicroscope and shown as an image at 1070, and a second color of stainedtissue imaged by a confocal microscope and shown as an image at 1080. Adisplayed rendered composite 3D image of the multi-channel 3D volumetricdata is shown at 1090.

Volume rendering of any of the data sets is performed at 1010 andiso-surface rendering is performed at 1011. Frame compositing andbuffering of rendered image data is performed at 1012.

In summary, systems and methods for the real time display, quantitationand manipulation of extremely large data sets using real timevolume/surface rendering and display of volumetric data images (e.g.,stereoscopic volumetric images) using all the acquired data fromextremely large data sets are disclosed. The systems and methods providethe ability to volumetrically render images with extremely highresolution in applications such as medical imaging procedures, digitalmicroscopy such as in use of a confocal microscope, and other areaswhere extremely large data sets are produced from the imaging process.

While the invention has been described with reference to certainembodiments, it will be understood by those skilled in the art thatvarious changes may be made and equivalents may be substituted withoutdeparting from the scope of the invention. In addition, manymodifications may be made to adapt a particular situation or material tothe teachings of the invention without departing from its scope.Therefore, it is intended that the invention not be limited to theparticular embodiment disclosed, but that the invention will include allembodiments falling within the scope of the appended claims.

1. A stand-alone platform providing real time high resolution imageprocessing, said stand-alone platform comprising: a first processingunit adapted to automatically read a plurality of volumetric data setscorresponding to a plurality of multi-modal or multi-channel volumetricdata formats derived from at least one volumetric data source andextract fully acquired volumetric data from said data sets; a system busoperationally connected to said first processing unit; a graphicsprocessing unit memory operationally connected to said system bus andadapted to receive said fully acquired volumetric data from said firstprocessing unit via said system bus; at least one graphics processingunit operationally connected to said graphics processing unit memory andadapted to render multi-dimensional image data from said fully acquiredvolumetric data in real time; and a frame compositing and bufferingdevice operationally connected to said at least one graphics processingunit and adapted to buffer frames of said rendered multi-dimensionalimage data and output said buffered frames for display in real time. 2.The stand-alone platform of claim 1 wherein said first processing unitis further adapted to co-register multi-modal fully acquired volumetricdata to a common spatial coordinate system before passing saidmulti-modal fully acquired volumetric data to said graphics processingunit memory.
 3. The stand-alone platform of claim 1 further comprising anetwork interface operationally connected to said system bus and adaptedto communicatively interface with a remote storage device and a remotedisplay subsystem.
 4. The stand-alone platform of claim 1 furthercomprising a first memory operationally connected to said firstprocessing unit via said system bus.
 5. The stand-alone platform ofclaim 1 further comprising a local storage device operationallyconnected to said system bus and adapted to load in said plurality ofvolumetric data sets from said at least one volumetric data source. 6.The stand-alone platform of claim 1 wherein said renderedmulti-dimensional image data comprises stereoscopic three-dimensional(3D) image data.
 7. A system providing multi-dimensional rendering,display, and manipulation of full high resolution volumetric data sets,said system comprising: a stand-alone platform adapted to rendermulti-dimensional image data in real time from a plurality of fullyacquired volumetric data sets corresponding to a plurality ofmulti-modal or multi-channel volumetric data formats derived from atleast one volumetric data source; at least one display subsystemoperationally connected to said stand-alone platform and adapted todisplay said rendered multi-dimensional image data in real time; and atleast one human interface device (HID) operationally connected to saidstand-alone platform and adapted to provide interactive real timemanipulation and modification of said rendered multi-dimensional imagedata.
 8. The system of claim 7 further comprising active stereoscopiceyewear wirelessly connected to and synchronized with said displaysubsystem and adapted to provide active stereoscopic viewing of saiddisplayed rendered multi-dimensional image data in real time to a userwearing said stereoscopic eyewear.
 9. The system of claim 7 furthercomprising passive stereoscopic eyewear adapted to provide passivestereoscopic viewing of said displayed rendered multi-dimensional imagedata in real time to a user wearing said passive stereoscopic eyewear.10. The system of claim 7 wherein said HID is further adapted to provideaccess to perform interactive real time quantitative analysis of saidrendered multi-dimensional image data.
 11. A system providingmulti-dimensional rendering, display, and manipulation of full highresolution volumetric data sets, said system comprising: a stand-aloneplatform adapted to render multi-dimensional image data in real timefrom a plurality of fully acquired volumetric data sets corresponding toa plurality of multi-modal or multi-channel volumetric data formatsderived from at least one volumetric data source; at least one remotethin client viewer operationally connected to said stand-alone platformand adapted to receive said rendered multi-dimensional image data fromsaid stand-alone platform in real time using a remote frame buffer (RFB)protocol, and adapted to stereoscopically display said renderedmulti-dimensional image data in real time; and at least one humaninterface device (HID) operationally connected to said remote thinclient viewer and adapted to initiate interactive real time manipulationand modification of said rendered multi-dimensional image data uponactivation of said HID by a user, wherein said manipulation andmodification is accomplished when said remote thin client viewer sendsan event command to said stand-alone platform in real time in responseto said HID activation, said rendered multi-dimensional image data isupdated by said stand-alone platform in real time in response to saidevent command, said stand-alone platform sends said updated renderedmulti-dimensional image data to said remote thin client viewer in realtime using said RFB protocol, and said updated renderedmulti-dimensional image data is stereoscopically displayed by saidremote thin client viewer in real time.
 12. The system of claim 11further comprising stereoscopic eyewear wirelessly connected to ansynchronized with said remote thin client viewer and adapted to providestereoscopic viewing of said displayed rendered multi-dimensional imagedata in real time to a user wearing said stereoscopic eyewear.
 13. Thesystem of claim 11 wherein said HID is further adapted to initiateinteractive real time quantitative analysis of said renderedmulti-dimensional image data upon further activation of said HID by auser, wherein said quantitative analysis is accomplished when saidremote thin client viewer sends an further event command to saidstand-alone platform in real time in response to said further activationof said HID, said quantitative analysis is performed by said stand-aloneplatform in real time in response to said further event command togenerate quantitative metrics, said stand-alone platform sends saidquantitative metrics to said remote thin client viewer in real timeusing said RFB protocol, and said quantitative metrics is displayed bysaid remote thin client viewer along with said renderedmulti-dimensional image data in real time.
 14. A method for themulti-dimensional rendering, display, manipulation, and analysis of fullhigh resolution volumetric data sets, said method comprising:automatically loading a plurality of fully acquired volumetric data setscorresponding to a plurality of multi-modal or multi-channel volumetricdata formats derived from at least one volumetric data source into astand-alone platform; processing said plurality of fully acquiredvolumetric data sets within said stand-alone platform to extract fullyacquired volumetric data from said data sets and to generate at leastone rendered volume image in real time from said extracted data;displaying said at least one rendered volume image on a displaysubsystem in real time; modifying at least one display parameter of saiddisplayed rendered volume image using at least one human interfacedevice (HID) operationally connected to said stand-alone platform;updating said at least one rendered volume image within said stand-aloneplatform in real time in response to said at least one modified displayparameter; and displaying said updated at least one rendered volumeimage on said display subsystem in real time.
 15. The method of claim 14wherein said at least one rendered volume image comprises a stereoscopicpair of volumetric images.
 16. The method of claim 14 wherein saidmodifying at least one display parameter results in at least one of: anorientation change of said at least one rendered volume image; a colortransfer function change of said at least one rendered volume image; anopacity transfer function change of said at least one rendered volumeimage; a segmentation parameter of said at least one rendered volumeimage; and an iso-surface generation parameter.
 17. The method of claim14 wherein said processing includes registering said extracted fullyacquired volumetric data to a common spatial coordinate system.
 18. Themethod of claim 14 further comprising selecting and displaying aniso-surface of said at least one rendered volume image in real time. 19.The method of claim 14 further comprising performing real timeinteractive quantitative analysis of said rendered volume image.
 20. Themethod of claim 14 further comprising segmenting a portion of saidrendered volume image in real time.
 21. The method of claim 20 furthercomprising computing a volume of said segmented portion in real time.22. The method of claim 20 further comprising computing a surface areaof said segmented portion in real time.
 23. A stand-alone platformproviding real time high resolution image processing, said stand-aloneplatform comprising: means for automatically reading a plurality ofvolumetric data sets corresponding to a plurality of multi-modal ormulti-channel volumetric data formats derived from at least onevolumetric data source and extracting fully acquired volumetric datafrom said data sets; means for rendering multi-dimensional image datafrom said fully acquired volumetric data in real time; and means forbuffering frames of said rendered multi-dimensional image data andoutputting said buffered frames for display in real time.
 24. Thestand-alone platform of claim 23 further comprising means forpre-processing said extracted fully acquired volumetric data beforeperforming said rendering.
 25. The stand-alone platform of claim 24wherein said pre-processing includes at least one of data clipping, datasegmenting, data deconvolution, data resampling, data normalizing, anddata converting.